Prognostic and diagnostic AI-based medical devices hold immense promise for advancing healthcare, yet their rapid development has outpaced the establishment of appropriate validation methods. Existing approaches often fall short in addressing the complexity of practically deploying these devices and ensuring their effective, continued operation in real-world settings. Building on recent discussions around the validation of AI models in medicine and drawing from validation practices in other fields, a framework to address this gap is presented. It offers a structured, robust approach to validation that helps ensure device reliability across differing clinical environments. The primary challenges to device performance upon deployment are discussed while highlighting the impact of changes related to individual healthcare institutions and operational processes. The presented framework emphasizes the importance of repeating validation and fine-tuning during deployment, aiming to mitigate these issues while being adaptable to challenges unforeseen during device development. The framework is also positioned within the current US and EU regulatory landscapes, underscoring its practical viability and relevance considering regulatory requirements. Additionally, a practical example demonstrating potential benefits of the framework is presented. Lastly, guidance on assessing model performance is offered and the importance of involving clinical stakeholders in the validation and fine-tuning process is discussed.

Purpose: Artificial intelligence (AI) solutions for medical diagnosis require thorough evaluation to demonstrate that performance is maintained for all patient sub-groups and to ensure that proposed improvements in care will be delivered equitably. This study evaluates the robustness of an AI solution for the diagnosis of normal chest X-rays (CXRs) by comparing performance across multiple patient and environmental subgroups, as well as comparing AI errors with those made by human experts. Methods: A total of 4,060 CXRs were sampled to represent a diverse dataset of NHS patients and care settings. Ground-truth labels were assigned by a 3-radiologist panel. AI performance was evaluated against assigned labels and sub-groups analysis was conducted against patient age and sex, as well as CXR view, modality, device manufacturer and hospital site. Results: The AI solution was able to remove 18.5% of the dataset by classification as High Confidence Normal (HCN). This was associated with a negative predictive value (NPV) of 96.0%, compared to 89.1% for diagnosis of normal scans by radiologists. In all AI false negative (FN) cases, a radiologist was found to have also made the same error when compared to final ground-truth labels. Subgroup analysis showed no statistically significant variations in AI performance, whilst reduced normal classification was observed in data from some hospital sites. Conclusion: We show the AI solution could provide meaningful workload savings by diagnosis of 18.5% of scans as HCN with a superior NPV to human readers. The AI solution is shown to perform well across patient subgroups and error cases were shown to be subjective or subtle in nature.

Finite State this week has added a binary analysis capability that enables device manufacturers to more easily identify zero-day vulnerabilities in software. Jeff Martin, vice president of product for Finite State, said this latest addition to the company's risk analysis platform can quickly assess third-party components for zero-day vulnerabilities and other known common vulnerabilities and exposures (CVEs). The Finite State risk analysis platform is primarily used by device manufacturers that typically employ on-board support packages (BSPs) and software development kits (SDKs) from third-party vendors and developers. The challenge they face is those BSPs and SDKs are essentially a black box that device manufacturers can't see inside, added Martin. In the absence of that visibility, Martin said device manufactures have no idea whether or not their software supply chains have been compromised by a zero-day vulnerability.

On October 14, 2021, the U.S. Food and Drug Administration ("FDA" or the "Agency") held a virtual workshop entitled, Transparency of Artificial Intelligence ("AI")/Machine Learning ("ML")-enabled Medical Devices. The workshop builds upon previous Agency efforts in the AI/ML space. Back in 2019, FDA issued a discussion paper and request for feedback called, Proposed Regulatory Framework for Modifications to AI/ML-Based Software as a Medical Device ("SaMD"). To support continued framework development and to increase collaboration and innovation between key stakeholders and specialists, FDA created the Digital Health Center of Excellence in 2020. And, in January 2021, FDA published an AI/ML Action Plan, based, in part, on stakeholder feedback to the 2019 discussion paper.

While artificial intelligence (AI) has the potential to revolutionize a number of industries, the technology isn't without its controversies. Over the past few years, researchers and developers have raised concerns around the potential impacts of widespread AI adoption--and how a lack of existing ethical frameworks may put consumers at risk. These concerns may be especially relevant to medical device manufacturers, which are increasingly using AI in new medical devices like smart monitors and health wearables. New standards and regulations on ethical AI may provide essential guidance for medical device manufacturers interested in leveraging AI. The widespread use of AI could pose a number of ethical challenges.

Where does your enterprise stand on the AI adoption curve? Take our AI survey to find out. The health care industry is starting to adopt digital twins to improve personalized medicine, health care organization performance, and new medicines and devices. Although simulations have been around for some time, today's medical digital twins represent an important new take. These digital twins can create useful models based on information from wearable devices, omics, and patient records to connect the dots across processes that span patients, doctors, and health care organizations, as well as drug and device manufacturers.

Companies are looking to grab any technology-driven advantage they can as they adapt to new ways of working, managing employees, and serving customers. They are making bigger moves toward the cloud, e-commerce, digital supply chains, artificial intelligence (AI) and machine learning (ML), data analytics, and other areas that can deliver efficiency and innovation. At the same time, enterprises are trying to manage risk -- and the same digital initiatives that create new opportunities can also lead to risks such as security breaches, regulatory compliance failures, and other setbacks. The result is an ongoing conflict between the need to innovate and the need to mitigate risk. "There is always going to be some amount of tension relating to managing risk and engaging in digital transformation work," says Ryan Smith, CIO at healthcare provider Intermountain Healthcare.

Edge AI is here to stay! Artificial intelligence (AI) is powering many real-world applications which we see in our daily lives. AI, once seen as an emerging technology, has now successfully penetrated into every industry (B2B & B2C) Banking, logistics, healthcare, defence, manufacturing, retail, automotive, consumer electronics. Smart Speaker like Echo, Google Nest, is one such example of Edge AI solutions in the consumer electronics sector. AI technology is powerful, and human-kind has set its eye on the path of harnessing its potential to the fullest. Intelligence brought to the device can be very useful and creative.

In a constantly changing world where virtual and physical environments are converging, ever-evolving digital technologies continue to disrupt the medical device industry. Besides, an uncertain environment, resulting from the ongoing COVID-19 outbreak has forced healthcare device manufacturers to adopt advanced technologies and tap into enormously growing medical device markets. At the same time, the increased connectivity between medical devices, physicians and consumers is necessitating OEMs to design and develop technologically advanced medical solutions. The digitization of manufacturing is, thus considered to be one of the core adjustments for healthcare manufacturers and OEMs to provide better patient care and remain competitive in the medical device markets. Advanced technologies like IoT, AI/ML, cloud, RPA, augmented and virtual reality hold a great potential to revolutionize the medical device manufacturing processes.

Earl Yardley, director at Industrial Vision Systems, writes about the recent impact of automation on the factory floor. Integrated quality inspection processes continue to make a significant contribution to medical device manufacturing production, including the provision of automated inspection capabilities as part of real-time quality control procedures. Long before COVID-19, medical device manufacturers were rapidly transforming their factory floors by leveraging technologies such as artificial intelligence (AI), machine vision, robotics, and deep learning. These investments have enabled them to continue to produce critical and high-demand products during these current times, even ramping up production to help address the pandemic. Medical device manufacturers must be lean, with high-speeds, and an ability to switch product variants quickly and easily, all validated to'Good Automated Manufacturing Practice' (GAMP).